Skip to content

Add filtering for kNN vector indexer test scenarios #130751

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Jul 8, 2025
Merged

Add filtering for kNN vector indexer test scenarios #130751

merged 2 commits into from
Jul 8, 2025

Conversation

@benwtrent
Copy link
Member

@benwtrent benwtrent commented Jul 7, 2025

This adds random filter search scenarios to our knn vector testing infrastructure.

The seed is by default static, and if the same seed and filter criteria is used over files older than then stored nearest neighbors, the stored nearest neighbors will be used.

Here is an example output:

index_name                      index_type  n_probe  latency(ms)  net_cpu_time(ms)  avg_cpu_count      QPS  recall   visited  filter_selectivity
------------------------------  ----------  -------  -----------  ----------------  -------------  -------  ------  --------  ------------------
cohere-wikipedia-docs-768d.vec         ivf       10         0.94              0.00           0.00  1063.83    0.69   2785.63                1.00
cohere-wikipedia-docs-768d.vec         ivf       20         1.10              0.00           0.00   909.09    0.76   5019.65                1.00
cohere-wikipedia-docs-768d.vec         ivf       30         1.44              0.00           0.00   694.44    0.81   7487.88                1.00
cohere-wikipedia-docs-768d.vec         ivf       40         1.67              0.00           0.00   598.80    0.84  10101.49                1.00
cohere-wikipedia-docs-768d.vec         ivf       50         1.89              0.00           0.00   529.10    0.86  12599.47                1.00
cohere-wikipedia-docs-768d.vec         ivf       60         1.97              0.00           0.00   507.61    0.88  15083.89                1.00
cohere-wikipedia-docs-768d.vec         ivf       70         2.35              0.00           0.00   425.53    0.89  17746.02                1.00
cohere-wikipedia-docs-768d.vec         ivf       80         2.82              0.00           0.00   354.61    0.90  20430.24                1.00
cohere-wikipedia-docs-768d.vec         ivf       90         2.29              0.00           0.00   436.68    0.91  23242.56                1.00
cohere-wikipedia-docs-768d.vec         ivf      100         2.51              0.00           0.00   398.41    0.92  26001.28                1.00
cohere-wikipedia-docs-768d.vec         ivf      200         4.41              0.00           0.00   226.76    0.95  52559.51                1.00
cohere-wikipedia-docs-768d.vec         ivf      200         4.35              0.00           0.00   229.89    0.95  52559.51                1.00
cohere-wikipedia-docs-768d.vec         ivf      200         4.42              0.00           0.00   226.24    0.95  52559.51                1.00
cohere-wikipedia-docs-768d.vec         ivf       10         1.16              0.00           0.00   862.07    0.73   2245.31                0.40
cohere-wikipedia-docs-768d.vec         ivf       20         1.40              0.00           0.00   714.29    0.79   3318.13                0.40
cohere-wikipedia-docs-768d.vec         ivf       30         1.66              0.00           0.00   602.41    0.82   4711.59                0.40
cohere-wikipedia-docs-768d.vec         ivf       40         2.05              0.00           0.00   487.80    0.85   6227.79                0.40
cohere-wikipedia-docs-768d.vec         ivf       50         2.32              0.00           0.00   431.03    0.87   7721.94                0.40
cohere-wikipedia-docs-768d.vec         ivf       60         2.70              0.00           0.00   370.37    0.88   9267.13                0.40
cohere-wikipedia-docs-768d.vec         ivf       70         2.85              0.00           0.00   350.88    0.90  10917.60                0.40
cohere-wikipedia-docs-768d.vec         ivf       80         3.41              0.00           0.00   293.26    0.91  12580.96                0.40
cohere-wikipedia-docs-768d.vec         ivf       90         3.62              0.00           0.00   276.24    0.92  14340.31                0.40
cohere-wikipedia-docs-768d.vec         ivf      100         4.00              0.00           0.00   250.00    0.93  16100.42                0.40
cohere-wikipedia-docs-768d.vec         ivf      200         7.61              0.00           0.00   131.41    0.97  33614.36                0.40
cohere-wikipedia-docs-768d.vec         ivf      200         7.69              0.00           0.00   130.04    0.97  33614.36                0.40
cohere-wikipedia-docs-768d.vec         ivf      200         7.98              0.00           0.00   125.31    0.97  33614.36                0.40

@elasticsearchmachine elasticsearchmachine added the Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch label Jul 7, 2025
@elasticsearchmachine
Copy link
Collaborator

elasticsearchmachine commented Jul 7, 2025

Pinging @elastic/es-search-relevance (Team:Search Relevance)

iverase
iverase approved these changes Jul 8, 2025
View reviewed changes
Copy link
Contributor

@iverase iverase left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

At some point we should add a readme file or a help option that explain all the possible inputs.

john-wagster
john-wagster approved these changes Jul 8, 2025
View reviewed changes
Copy link
Contributor

@john-wagster john-wagster left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@benwtrent benwtrent merged commit c7c5d4a into elastic:main Jul 8, 2025
33 checks passed
@benwtrent benwtrent deleted the add-filter-knn-index-tester branch July 8, 2025 14:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@iverase iverase iverase approved these changes

@john-wagster john-wagster john-wagster approved these changes

Assignees

No one assigned

Labels

>non-issue :Search Relevance/Vectors Vector search Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch v9.2.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@benwtrent @elasticsearchmachine @iverase @john-wagster